Papers with Neural Machine Translation
PENELOPIE: Enabling Open Information Extraction for the Greek Language through Machine Translation (2021.eacl-srw)
Copied to clipboard
| Challenge: | a new method for information extraction from Greek corpora is being developed for low-resource languages. |
| Approach: | They propose a methodology that aims at bridging the gap between high and low-resource languages in the context of Open Information Extraction. |
| Outcome: | The proposed method outperforms the current state-of-the-art for the Greek language on benchmark datasets. |
Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Models (2025.tacl-1)
Copied to clipboard
| Challenge: | a recent study revisits six core challenges that have influenced the evolution of Neural Machine Translation (NMT) domain mismatch, amount of parallel data, rare word prediction, translation of long sentences and sub-optimal beam search remain challenges in LLMs. |
| Approach: | They revisit core challenges that have acted as benchmarks for progress in NMT . they propose to revisit these challenges and offer insights into their relevance . |
| Outcome: | The proposed models significantly improve translation of sentences containing approximately 80 words, even translating documents up to 512 words. |
A Simple and Fast Strategy for Handling Rare Words in Neural Machine Translation (2022.aacl-srw)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) has been gaining popularity due to its ability to bias in highfrequency words, low-frequency words have little chance of being considered in the inference process. |
| Approach: | They propose a strategy for integrating constraints during the training and decoding process to improve the translation of rare words. |
| Outcome: | The proposed approach improves translation of rare words in high and low-resource translation tasks, showing improvements of up to +1.8 BLEU scores over baseline systems. |
Triangular Architecture for Rare Language Translation (P18-1)
Copied to clipboard
| Challenge: | Empirical results show that Neural Machine Translation (NMT) performs poor on low-resource pairs especially when Z is a rare language. |
| Approach: | They propose a triangular triangulation technique to leverage bilingual data to optimize the translation performance of low-resource pairs. |
| Outcome: | Empirical results show that the proposed architecture significantly improves translation quality of rare languages on MultiUN and IWSLT2012 datasets and even better when combining back-translation methods. |
ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MT (2025.acl-short)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) has improved translation by using Transformer-based models, but still struggles with word ambiguity and context. |
| Approach: | They create a new Czech-to-polish e-commerce product translation dataset coupled with images and product metadata consisting of 11,400 sentence pairs. |
| Outcome: | The proposed model incorporates visual cues alongside textual data to improve translation quality. |
Combining Subword Representations into Word-level Representations in the Transformer Architecture (2020.acl-srw)
Copied to clipboard
| Challenge: | Currently dominant approaches use word-level tokens, but this increases the length of the sequences and makes it difficult to profit from word-based information. |
| Approach: | They propose to combine subword-level representations into word-level ones in the first layers of the encoder, reducing the effective length of the sequences in the following layers. |
| Outcome: | The proposed model maintains translation quality with no extra word-level information . it is superior to the current dominant method for incorporating word- level source language information a priori . |
Addressing the Vulnerability of NMT in Input Perturbations (2021.naacl-industry)
Copied to clipboard
| Challenge: | Recent advances in NMT have improved translation quality but are vulnerable to input perturbations. |
| Approach: | They propose a method to reduce the effect of noisy inputs by using a Context-Enhanced Reconstruction approach. |
| Outcome: | The proposed approach improves robustness on Chinese-English and French-English translation tasks. |
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing (D18-2)
Copied to clipboard
| Challenge: | Existing subword segmentation tools assume input is pre-tokenized into word sequences, but SentencePiece can train subword models directly from raw sentences. |
| Approach: | They propose a language-independent subword tokenizer and detokenizer for Neural-based text processing. |
| Outcome: | The proposed system achieves comparable accuracy to training from raw sentences. |
Thesis proposal: Are We Losing Textual Diversity to Natural Language Processing? (2026.eacl-srw)
Copied to clipboard
| Challenge: | Using Neural Machine Translation, we examine whether the algorithms used in NMT have inherent inductive biases that are beneficial for most types of inputs but might harm the processing of untypical texts. |
| Approach: | They propose to use a set of measures to quantify text diversity based on its statistical properties to determine whether NMT systems struggle with maintaining the diversity of such texts. |
| Outcome: | The proposed approaches maintain the diversity and complexity of language and allow for better global planning of the output generation. |
Combining Translation Memory with Neural Machine Translation (D19-52)
Copied to clipboard
| Challenge: | Existing systems that combine translation memory and statistical machine translation (MT) models are able to translate less familiar phrases and sentences without sacrificing quality. |
| Approach: | They propose to combine translation memory and Neural Machine Translation (NMT) models to select final translation outputs when similarity score of a test source sentence exceeds the predefined threshold. |
| Outcome: | The proposed system significantly improves translation performance on the Timely Disclosure corpus, as compared to a standalone NMT system. |
Converting the Point of View of Messages Spoken to Virtual Assistants (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Using a voice message, virtual assistants extract the message and send it to the user’s contact, rather than properly converting it to “I love you.” |
| Approach: | They propose to take a voice message from one user, convert it to “I love you” and deliver it to its target user. |
| Outcome: | The proposed system can take a voice message from one user, convert the point of view of the message, and then deliver the result to its target user. |
Reference Free Domain Adaptation for Translation of Noisy Questions with Question Specific Rewards (2023.findings-emnlp)
Copied to clipboard
Baban Gain, Ramakrishna Appicharla, Soumya Chennabasavaraj, Nikesh Garera, Asif Ekbal, Muthusamy Chelliah
| Challenge: | Creating a synthetic parallel corpus from noisy data is also difficult due to its noisy nature. |
| Approach: | They propose a training methodology that fine-tunes the NMT system only using source-side data to balance adequacy and fluency. |
| Outcome: | The proposed method surpasses the MLE-based fine-tuning approach by achieving a 1.9 BLEU improvement. |
Adaptively Scheduled Multitask Learning: The Case of Low-Resource Neural Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Neural Machine Translation suffers from the lack of bilingual data in low-resource scenarios. |
| Approach: | They propose to inject inductive biases into Neural Machine Translation (NMT) using auxiliary syntactic and semantic tasks. |
| Outcome: | The proposed approach improves translation quality by reweighing training data of main and auxiliary tasks based on their contributions to generalisability of main task. |
Data Pruning for Efficient Model Pruning in Neural Machine Translation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models have demonstrated encouraging performance in various NLP tasks at the cost of over-parametrized networks and high memory requirements. |
| Approach: | They combine data pruning with movement pruning for Neural Machine Translation to enable efficient fine-pruning by leveraging cross-entropy scores of individual training instances. |
| Outcome: | The proposed pruning strategy outperforms other pruning methods on a translation task and shows that training cross-entropy scores can reduce the steps required for convergence and training time. |
Cultural and Geographical Influences on Image Translatability of Words across Languages (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural machine translation models produce poor translations when there are few/no parallel sentences to train the models. |
| Approach: | They define image translatability as the translability of words as images associated with words in different languages that have a high degree of visual similarity. |
| Outcome: | The proposed model improves upon text-only models only marginally. |
An Effective Approach to Unsupervised Machine Translation (P19-1)
Copied to clipboard
| Challenge: | a recent research line has managed to train both unsupervised and unsupervised machine translation systems using monolingual corpora only. |
| Approach: | They propose to use monolingual corpora to train both unsupervised and unsupervised machine translation systems. |
| Outcome: | The proposed system achieves 22.5 BLEU points in English-to-German WMT 2014, 5.5 points more than the previous best unsupervised system, and 0.5 points more in the (supervised) shared task winner back in 2014. |
On the Importance of Word Boundaries in Character-level Neural Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Neural Machine Translation models typically use a fixed-size lexical vocabulary . subword segmentation methods rely on statistical heuristics that lack any linguistic notion . |
| Approach: | They propose a hierarchical decoding architecture for character-level NMT using subwords . they propose fewer parameters and a more efficient approach to perform translation at the level of words . |
| Outcome: | The proposed model can reach higher translation accuracy than the subword-level model with fewer parameters while maintaining longer-distance contextual and grammatical dependencies. |
Our Neural Machine Translation Systems for WAT 2019 (D19-52)
Copied to clipboard
| Challenge: | In the last five years, statistical machine translation is gradually fading out in favor of neural machine translation. |
| Approach: | They describe a novel Neural Machine Translation (NMT) system for the WAT 2019 translation tasks they focus on. |
| Outcome: | The proposed system improves translation accuracy while replacing absolute position representations with relative positions. |
KIT Lecture Translator: Multilingual Speech Translation with One-Shot Learning (C18-2)
Copied to clipboard
Florian Dessloch, Thanh-Le Ha, Markus Müller, Jan Niehues, Thai-Son Nguyen, Ngoc-Quan Pham, Elizabeth Salesky, Matthias Sperber, Sebastian Stüker, Thomas Zenkel, Alexander Waibel
| Challenge: | In today's globalized world, communication is difficult and often the language barrier still prevents communication. |
| Approach: | They have developed a low-latency translation system that is adapted to lectures and covers several language pairs. |
| Outcome: | The proposed system improves performance but also covers several European languages. |
Efficient Neural Machine Translation for Low-Resource Languages via Exploiting Related Languages (2020.acl-srw)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) is a rapidly advancing MT paradigm that can be used to improve machine translation for many languages. |
| Approach: | They propose a technique called Unified Transliteration and Subword Segmentation to leverage language similarity while exploiting parallel data from related languages. |
| Outcome: | The proposed approach improves translation accuracy by 5 BLEU points over the standard Transformer-based NMT models. |
A Copy Mechanism for Handling Knowledge Base Elements in SPARQL Neural Machine Translation (2022.findings-aacl)
Copied to clipboard
| Challenge: | Current architectures are unable to integrate knowledge base schema and handle questions unseen during training rendering them unusable outside the scope of topics covered in the training set. |
| Approach: | They propose to integrate a copy mechanism for neural SPARQL query generation by adding a knowledge base layer and a dynamic knowledge base vocabulary to two Seq2Seq architectures. |
| Outcome: | The proposed model outperforms existing models on state-of-the-art datasets and shows a significant increase in performance. |
NLPRL at WAT2019: Transformer-based Tamil – English Indic Task Neural Machine Translation System (D19-52)
Copied to clipboard
| Challenge: | a majority of Asians speak low to medium resource languages . lack of resources poses a challenge, which requires innovative solutions . |
| Approach: | They propose a Neural Machine Translation system for Tamil-English Indic Task . they train a system for both Tamil-to-English and English-to Tamil pairs . |
| Outcome: | The proposed system is based on a Transformer-based architecture and is not very innovative, but can be treated as an incremental step in this direction. |
Domain Adaptive Inference for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation models are effective when trained on broad domains with large datasets, such as news translation. |
| Approach: | They propose a novel approach for adaptive ensemble weighting for Neural Machine Translation by extending Bayesian Interpolation with source information. |
| Outcome: | The proposed approach improves performance on Spanish-English and English-German tasks without the need for the domain label. |
Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) currently exhibits biases such as producing translations that are too short and overgenerating frequent words. |
| Approach: | They propose to use minimum bayes risk decoding instead of beam search to investigate the effects of beam decoding on unbiased samples. |
| Outcome: | The proposed method improves on a number of previously reported biases and failure cases of beam search on unbiased samples. |
Why Find the Right One? (2021.eacl-srw)
Copied to clipboard
| Challenge: | a new study investigates the impact of anaphoric one words in English on the neural machine translation process. |
| Approach: | They investigate the impact of anaphoric one words in English on the Neural Machine Translation process using English-Hindi as source and target language pair. |
| Outcome: | The proposed system performs poorly on sentences containing anaphoric ones compared to sentences involving regular, non-anaphorical ones . the results show that amongst the anamorphic words, the noun class is clearly much harder for NMT than the determinatives . |
Neural Machine Translation for Agglutinative Languages via Data Rejuvenation (2025.acl-srw)
Copied to clipboard
| Challenge: | Recent years, advances in Neural Machine Translation (NMT) heavily rely on large-scale parallel corpora. |
| Approach: | They propose to combine fine-grained inactive sample identification with target-side rejuvenation to improve translation quality from agglutinative languages. |
| Outcome: | The proposed framework improves on four low-resource agglutinative language tasks. |
A Semantic Uncertainty Sampling Strategy for Back-Translation in Low-Resources Neural Machine Translation (2025.acl-srw)
Copied to clipboard
Yepai Jia, Yatu Ji, Xiang Xue, Shilei@imufe.edu.cn Shilei@imufe.edu.cn, Qing-Dao-Er-Ji Ren, Nier Wu, Na Liu, Chen Zhao, Fu Liu
| Challenge: | Back-translation methods rely on large-scale parallel corpora to enhance performance, but ignore the semantic quality of monolingual data. |
| Approach: | They propose a method which prioritizes sentences with higher semantic uncertainty as training samples by computationally evaluating the complexity of unannotated monolingual data. |
| Outcome: | The proposed method improves translation accuracy and fluency by +1.7 on all three translation tasks. |
Addressing Troublesome Words in Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has weaknesses in handling lowfrequency and ambiguous words, which we refer to as troublesome words. |
| Approach: | They propose to use contextual memory to memorize which target words should be produced in which situations to translate troublesome words. |
| Outcome: | The proposed method outperforms baseline models on Chinese-to-English and English-to German translation tasks. |
Learning Source Phrase Representations for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to machine translation have been shown to be effective for long sentences . however, the attentional network can't capture long-distance dependencies . |
| Approach: | They propose a multi-head attention mechanism which generates phrase representations from token representations and incorporates them into the Transformer translation model to enhance its ability to capture long-distance relationships. |
| Outcome: | The proposed model can be computed in parallel and improves on the WMT 14 tasks. |
Top-down Tree Structured Decoding with Syntactic Connections for Neural Machine Translation and Parsing (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) models are based on sequential decoding or serialisation of structured data into sequence. |
| Approach: | They propose a model that combines sequential encoder with tree-structured decoding augmented with a syntax-aware attention model. |
| Outcome: | The proposed model produces fluent translations with better reordering than previous models. |
Checkpoint Reranking: An Approach to Select Better Hypothesis for Neural Machine Translation Systems (2020.acl-srw)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) has produced excellent results in the field of machine translation due to generation of high-quality translations for different language pairs. |
| Approach: | They propose a method of re-ranking the outputs of Neural Machine Translation systems by focusing on the decoder's ability to generate distinct tokens and without the use of any language model or data. |
| Outcome: | The proposed method achieves translation improvement up to +0.16 BLEU points over baseline. |
Attention Weights in Transformer NMT Fail Aligning Words Between Sequences but Largely Explain Model Predictions (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Using attention weights, we show that NMT models make alignment errors by relying on uninformative tokens from the source sequence. |
| Approach: | They propose to use attention weights to regulate alignment errors in NMT models . they propose methods that largely reduce the word alignment error rate compared to standard induced alignments from attention weighted tokens. |
| Outcome: | The proposed methods reduce the word alignment error rate compared to standard induced alignments from attention weights. |
Multiscale Collaborative Deep Models for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Neural machine translation models with deeper neural networks are difficult to train. |
| Approach: | They propose a MultiScale Collaborative framework to boost gradient back-propagation . they let each encoder block learn a fine-grained representation and enhance it . |
| Outcome: | The proposed framework outperforms baseline models on translation tasks with three translation directions and achieves a BLEU score of 30.56 on the English-to-German task. |
Back-Translation Sampling by Targeting Difficult Words in Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) uses a sequence-to-sequence model to generate synthetic data. |
| Approach: | They propose a method that adds synthetic data to sentences with high prediction loss during training and a variety of sampling strategies targeting difficult-to-predict words. |
| Outcome: | The proposed method improves translation quality by up to 1.7 and 1.2 Bleu points over back-translation using random sampling for German-English and English-German, respectively. |
English-to-Chinese Transliteration with Phonetic Auxiliary Task (2020.aacl-main)
Copied to clipboard
| Challenge: | Several NMT techniques have been used to enhance machine transliteration models, but few focus on the linguistic features specific to the relevant languages. |
| Approach: | They propose a phonetic auxiliary task that integrates phonetic features into a model to improve generalization performance of the main transliteration task. |
| Outcome: | The proposed model achieves similar performance to the current state of the art with a much smaller size. |
Data Weighted Training Strategies for Grammatical Error Correction (2020.tacl-1)
Copied to clipboard
| Challenge: | Recent advances in the task of Grammatical Error Correction (GEC) have been driven by addressing data sparsity, both through new methods for generating large and noisy pretraining data and through the publication of small and higher-quality finetuning data in the BEA-2019 shared task. |
| Approach: | They propose to incorporate delta-log-perplexity, a type of example scoring, into a training schedule for Grammatical Error Correction (GEC) they perform experiments that shed light on the function and applicability of delta- log-perplicity. |
| Outcome: | The proposed methods incorporate delta-log-perplexity, a type of example scoring, into a training schedule for the task. |
M3T: A New Benchmark Dataset for Multi-Modal Document-Level Machine Translation (2024.naacl-short)
Copied to clipboard
Benjamin Hsu, Xiaoyu Liu, Huayang Li, Yoshinari Fujinuma, Maria Nadejde, Xing Niu, Ron Litman, Yair Kittenplon, Raghavendra Pappagari
| Challenge: | Document translation is a challenge for machine translation systems that focus on textual content at the sentence level, ignoring global context and visual layout structure. |
| Approach: | They propose a benchmark dataset to evaluate document-level NMT systems . they use visual cues to preserve reading order and contiguous blocks of text . |
| Outcome: | The proposed benchmarks assess document-level NMT systems on the comprehensive task of translating semi-structured documents. |
Control-DAG: Constrained Decoding for Non-Autoregressive Directed Acyclic T5 using Weighted Finite State Automata (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing non-autoregressive (NAR) models fail to generate specified entity names in up to 40% of responses and produce OOV errors. |
| Approach: | They propose a constrained decoding algorithm for Directed Acyclic T5 model which offers lexical, vocabulary and length control. |
| Outcome: | The proposed model significantly improves on Schema Guided Dialogue and DART datasets, establishing strong results for Task-Oriented Dialog and Data-to-Text NLG. |
Improving Neural Machine Translation Robustness via Data Augmentation: Beyond Back-Translation (D19-55)
Copied to clipboard
| Challenge: | Neural Machine Translation models are sensitive to noise in the input data. |
| Approach: | They propose new methods to extend limited noisy data and further improve NMT robustness to noise while keeping the models small. |
| Outcome: | The proposed methods extend limited noisy data and improve robustness to noise while keeping the models small. |
English-Indonesian Neural Machine Translation for Spoken Language Domains (P19-2)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is a data-driven method that requires a large amount of data to build a robust model. |
| Approach: | They conduct a study on Neural Machine Translation (NMT) for English-Indonesian and Indonesian-English (ID-EN) they build NMT systems using the Transformer model for both translation directions and implement domain adaptation method to train pre-trained NMT on speech language data. |
| Outcome: | The proposed model can learn formal translation outputs for English-Indonesian and Indonesian-English (ID-EN) given a small dataset of speech-styled language and a larger dataset of less formal language, the proposed model will be useful for learning formality level. |
Near Human-Level Performance in Grammatical Error Correction with Hybrid Machine Translation (N18-2)
Copied to clipboard
| Challenge: | Currently, most effective GEC systems are based on phrase-based statistical machine translation. |
| Approach: | They combine two of the most popular approaches to automated Grammatical Error Correction (GEC) they create a hybrid GEC system that preserves the accuracy of SMT output and generates more fluent sentences . |
| Outcome: | The proposed system achieves state-of-the-art on the CoNLL-2014 and JFLEG benchmarks. |
High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metrics (2022.tacl-1)
Copied to clipboard
| Challenge: | Neural machine translations are ranked below human translations in professional evaluations . |
| Approach: | They apply minimum bayes risk decoding to optimize different metrics of translation quality . they show that model estimates and translation quality only vaguely correlate . |
| Outcome: | The proposed method improves human translations with different models and metric. |
A Simple and Effective Approach to Coverage-Aware Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) models are used to solve translation problems using long-term models. |
| Approach: | They propose a method to seek a better balance between model confidence and length preference for Neural Machine Translation. |
| Outcome: | The proposed model improves on Chinese-English and English-German translation tasks. |
Multi-representation ensembles and delayed SGD updates improve syntax-based NMT (P18-2)
Copied to clipboard
| Challenge: | Existing studies have shown that NMT models trained to generate target syntax exhibit improved sentence structure relative to those trained on plain-text. |
| Approach: | They propose an approach to decoding ensembles of models generating different representations, focusing on models generating syntax. |
| Outcome: | The proposed approach gives state-of-the-art performance on a difficult Japanese-English task. |
Quality Does Matter: A Detailed Look at the Quality and Utility of Web-Mined Parallel Corpora (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing web-mined corpora for low-resource languages have serious quality issues, especially for lowresource language pairs. |
| Approach: | They ranked each corpus according to a similarity measure and evaluated different portions of this ranked corpus. |
| Outcome: | The results show that the quality of web-mined corpora for low-resource languages is significantly different from human-curated corporats. |
Align-to-Distill: Trainable Attention Alignment for Knowledge Distillation in Neural Machine Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing knowledge distillation approaches to NMT often rely on heuristics when deciding which teacher layers to distill from. |
| Approach: | They propose an approach to align student attention heads with their teacher counterparts by heuristics to solve a feature mapping problem. |
| Outcome: | The proposed strategy shows gains of +3.61 and +0.63 BLEU points for WMT-2022 DeDsb and WMT-2014 EnDe compared to baselines. |
A Classification-Guided Approach for Adversarial Attacks against Neural Machine Translation (2024.eacl-long)
Copied to clipboard
| Challenge: | Extensive research has been devoted to adversarial attacks against NMT models . perturbations of inputs can mislead the target model, resulting in incorrect outputs . |
| Approach: | They propose an adversarial attack framework that alters the class of output translations of an NMT model and a classifier to craft adversarials whose translations belong to a different class . |
| Outcome: | The proposed approach has a more substantial effect on the translation by altering the overall meaning, which leads to a different class determined by an oracle classifier. |
Exploring Enhanced Code-Switched Noising for Pretraining in Neural Machine Translation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Multilingual pretraining approaches to denoise synthetic code-switched data have shown that they generate the noise using non-contextual, one-to-one word translations obtained from lexicons. |
| Approach: | They propose an approach where contextual, many-to-many word translations are generated using a ‘base’ NMT model. |
| Outcome: | The proposed approach improves on 3 different language families and shows that small models can perform better than massive models like mBART50 and mRASP2 . |
Latent Part-of-Speech Sequences for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Existing methods for learning target side syntactic structure are greedy and only allow them to explore a limited portion of the latent space. |
| Approach: | They propose a new latent variable model, LaSyn, that captures the co-dependence between syntax and semantics while allowing for effective inference over the latent space. |
| Outcome: | The proposed model captures the co-dependence between syntax and semantics while allowing for efficient inference over the latent space. |
On the Difficulty of Translating Free-Order Case-Marking Languages (2021.tacl-1)
Copied to clipboard
| Challenge: | Free-order case-marking languages are more difficult to model than fixed-order languages . however, in medium- and low-resource settings, the overall NMT quality of fixed-or-fixed-order language pairs remains unmatched. |
| Approach: | They investigate whether certain languages are harder to model than others by adding case marking to their models. |
| Outcome: | The proposed models show that word order flexibility in the source language leads to a small loss of NMT quality even though the core verb arguments become impossible to disambiguate in sentences without semantic cues. |
Token-level Adaptive Training for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing token imbalance phenomenon in natural language as different tokens appear with different frequencies, which leads to different learning difficulties for tokens in Neural machine translation (NMT). |
| Approach: | They propose to assign tokens with different frequencies to target tokens during training to encourage the model to pay more attention to low-frequency tokens. |
| Outcome: | The proposed model yields consistent improvements on ZH-EN, EN-RO, and EN-DE translation tasks, especially on sentences that contain more low-frequency tokens. |
Multi-Unit Transformers for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that the MUTE models outperform the Transformer-Base by up to +1.52, +1.99 and +1.00 BLEU points, with only a mild drop in inference speed (about 3.1%). |
| Approach: | They propose to use multiple parallel units to promote the expressiveness of the Transformer by introducing diverse and complementary units. |
| Outcome: | The proposed models outperform the Transformer-Base model with only a mild drop in inference speed (about 3.1%). |
Multi-agent Learning for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Experimental results show that training with more than one agent improves translation quality and improves accuracy. |
| Approach: | They propose to introduce diverse agents in an in- teractive updating process to train NMT models with an additional agent. |
| Outcome: | The proposed approach improves on NIST Chinese-English, IWSLT 2014 German- English, WMT 2014 English-German translation tasks and shows competitive performance on all tasks. |
RealFormer: Transformer Likes Residual Attention (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing techniques to create Residual Attention Layer Transformer networks outperform the canonical Transformer on a wide spectrum of tasks. |
| Approach: | They propose a technique to create Residual Attention Layer Transformer networks that outperform the canonical Transformer on a wide spectrum of tasks. |
| Outcome: | The proposed technique outperforms the canonical Transformer on a wide spectrum of tasks including Masked Language Modeling, GLUE, SQUAD, Neural Machine Translation, WikiHop, HotpotQA, Natural Questions, and OpenKP. |
When and Why Are Pre-Trained Word Embeddings Useful for Neural Machine Translation? (N18-2)
Copied to clipboard
| Challenge: | Pre-trained word embeddings have proven to be invaluable for improving performance in natural language analysis tasks where large-scale parallel corpora cannot be obtained. |
| Approach: | They perform five sets of experiments to analyze when pre-trained word embeddings can be useful in NMT tasks. |
| Outcome: | The embeddings provide gains of up to 20 BLEU points in the most favorable setting. |
Do GPTs Produce Less Literal Translations? (2023.acl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are general-purpose language models capable of many natural language generation or understanding tasks. |
| Approach: | They investigate how LLMs differ qualitatively from standard Neural Machine Translation models by measuring literalness and monotonicity. |
| Outcome: | The proposed models achieve close to state-of-the-art translation performance under few-shot prompting . the results are backed up by human evaluations and a newer MT quality metrics . |
Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation models are influenced by two types of context, source and target, but none explicitly evaluates relative contribution to generation decision. |
| Approach: | They propose to adopt a variant of Layerwise Relevance Propagation which evaluates relative contributions to the generation decision by a proportion of token influence. |
| Outcome: | The proposed model can evaluate the relative contribution of source and target to the generation decision by using a variant of Layerwise Relevance Propagation (LRP) |
The Curious Case of Hallucinations in Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) suffers from well known pathologies such as coverage, mistranslation of named entities, etc. |
| Approach: | They propose a theory that explains hallucinations under source perturbation and a method that generates hallucines under corpus-level noise without any source perturbations. |
| Outcome: | The proposed hypothesis is validated by a corpus-level noise analysis and is validateable in other datasets. |
Adaptive Knowledge Sharing in Multi-Task Learning: Improving Low-Resource Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) requires large amounts of bilingual data to learn a translation model with reasonable quality. |
| Approach: | They propose to extend recurrent units with multiple "blocks" along with a trainable "routing network" this allows for adaptive collaboration by dynamic sharing of blocks conditioned on the task at hand, input, and model state. |
| Outcome: | Empirical evaluations of two low-resource translation tasks show +1 BLEU score improvements compared to strong baselines. |
Extract and Attend: Improving Entity Translation in Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve entity translation in Neural machine translation still suffer from inaccurate translation of entities due to the lack of entity training instances. |
| Approach: | They propose an extract-and-tend approach to enhance entity translation in NMT by extracting entities from a dictionary and attending to them with a prefix. |
| Outcome: | Experiments on En-Zh and En-Ru show that the proposed approach improves translation accuracy and translation quality. |
BitextEdit: Automatic Bitext Editing for Improved Low-Resource Machine Translation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to improve Neural Machine Translation (NMT) for lowresource languages are often trained on heuristically aligned or automatically mined data. |
| Approach: | They propose to filter out imperfect translations that yield unreliable training signals for Neural Machine Translation (NMT) instead, they propose to refine mined bitexts by automatic editing . |
| Outcome: | The proposed method improves the quality of mined bitexts for low-resource languages by up to 8 BLEU points. |
Data Selection Curriculum for Neural Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural Machine Translation models are typically trained on heterogeneous data that are concatenated and randomly shuffled. |
| Approach: | They propose a two-stage curriculum training framework where a NMT model is fine-tuned on subsets of data, selected by deterministic scoring and online scoring. |
| Outcome: | The proposed framework improves on six language pairs comprising low- and high-resource languages and shows up to +2.2 BLEU improvement and faster convergence. |
Incorporating Syntactic Uncertainty in Neural Machine Translation with a Forest-to-Sequence Model (C18-1)
Copied to clipboard
| Challenge: | Incorporating syntactic information in machine translation can lead to better reorderings, especially useful when the language pairs are syntaktically highly divergent. |
| Approach: | They propose a forest-to-sequence NMT model which uses exponentially many parse trees of the source sentence to compensate for parser errors. |
| Outcome: | The proposed model outperforms the sequence-to-sequence and tree-to tree-based models on English, Chinese and Farsi translation tasks. |
Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlations (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to train a multilingual NMT model for low-resource languages are lacking in terms of number of supervised examples. |
| Approach: | They propose to use decoder pre-training and back-translation to solve the degeneracy problem by analyzing spurious correlations between source and decoded sentences. |
| Outcome: | The proposed methods show significant improvement over the pivot-based approach on three challenging multilingual datasets. |
A Reinforcement Learning Approach to Improve Low-Resource Machine Translation Leveraging Domain Monolingual Data (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning domain adaptation have overfitting problem in low-resource domains . lack of parallel data makes it difficult for model to learn domain-specific knowledge . |
| Approach: | They propose a Reinforcement Learning Domain Adaptation method for Neural Machine Translation that uses in-domain source monolingual data to make up for the lack of parallel data. |
| Outcome: | The proposed method can alleviate overfitting and reinforce the model to learn domain-specific knowledge. |
PROSE: A Pronoun Omission Solution for Chinese-English Spoken Language Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Pro-drop (‘pronoun-dropping’) language requires NMT systems to recover omitted pronouns, but this task lacks sufficient datasets for benchmarking . |
| Approach: | They propose a benchmarking method that leverages the semantic embedding of dropped pronouns to augment training pairs to alleviate the negative impact introduced by pro-drop . |
| Outcome: | The proposed method outperforms existing methods regarding omitted pronoun retrieval and overall translation quality on four Chinese-English translation corpora. |
DaLC: Domain Adaptation Learning Curve Prediction for Neural Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Current research in NMT Domain Adaptation rarely provides insights on the amount of data required to perform Domain . |
| Approach: | They propose a Domain adaptation learning curve prediction model that predicts prospective DA performance based on in-domain monolingual samples in the source language. |
| Outcome: | The proposed model predicts prospective DA performance based on in-domain monolingual samples in the source language. |
English-Basque Statistical and Neural Machine Translation (L18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) requires large training corpora, which is problematic for low-resource languages. |
| Approach: | They propose to use an open-domain and an IT-domain corpora to train machine translations in English-Basque. |
| Outcome: | The proposed systems outperform OpenNMT, Moses SMT and Google Translate in English-Basque translation. |
Exploiting Pre-Ordering for Neural Machine Translation (L18-1)
Copied to clipboard
| Challenge: | Existing studies have shown that Neural Machine Translation suffers from the problems that some source words are mistakenly translated for multiple times . |
| Approach: | They propose a pre-ordering approach to solve the under-translation problem by pre-ordnanced source sentences and position embedding to enhance monotone translation. |
| Outcome: | The proposed method significantly improves translation quality by 2.43 BLEU points on Chinese-to-English translation. |
Boosting Neural Machine Translation with Similar Translations (2020.acl-main)
Copied to clipboard
| Challenge: | Statistical Machine Translation and fuzzy matching are completely different in their finality. |
| Approach: | They propose to use fuzzy matching to train neural machine translation to make use of similar translations, in a similar way a human translator employs fuzzy matches. |
| Outcome: | The proposed methods improve translation accuracy and fine-tuned model for unseen translation pairs. |
IndicBART: A Pre-trained Model for Indic Natural Language Generation (2022.findings-acl)
Copied to clipboard
| Challenge: | IndicBART is a multilingual, sequence-to-sequence pre-trained model focusing on 11 Indic languages and English. |
| Approach: | They present a multilingual sequence-to-sequence pre-trained model for Indic languages . they evaluate it on two NLG tasks: Neural Machine Translation and extreme summarization . |
| Outcome: | The proposed model performs well on low-resource translation scenarios . Script sharing, multilingual training, and better utilization contribute to the performance. |
Examining the Tip of the Iceberg: A Data Set for Idiom Translation (L18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) has been widely used in recent years with significant improvements for many language pairs. |
| Approach: | They propose to use a large-scale data set to evaluate idiom translation in GermanEnglish. |
| Outcome: | The proposed dataset is used to perform preliminary NMT experiments on idiom translation in GermanEnglish. |
Contextual Refinement of Translations: Large Language Models for Sentence and Document-Level Post-Editing (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models have demonstrated considerable success in various natural language processing tasks, but their performance in NMT tasks is still underexplored. |
| Approach: | They propose to use LLMs as automatic post-editors rather than direct translators to improve BLEU and COMET performance. |
| Outcome: | The proposed approach improves BLEU but COMET performance compared to in-context learning. |
Contrastive Decoding Reduces Hallucinations in Large Multilingual Machine Translation Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Hallucinations occur when the target side sentence is detached from the source side sentence, or in other words, when there is a low contribution of the source sentence to the generation of the target sentence. |
| Approach: | They propose to use Contrastive Decoding to maximise the log-likelihood difference between a model and the same model with reduced contribution from the encoder outputs. |
| Outcome: | The proposed algorithm maximises the log-likelihood difference between a model and the same model with reduced contribution from the encoder outputs. |
How Much Attention Do You Need? A Granular Analysis of Neural Machine Translation Architectures (P18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) has been replaced by convolutional or self-attentional approaches. |
| Approach: | They propose an architecture definition language that allows for a flexible combination of common building blocks. |
| Outcome: | The proposed architectures can bring recurrent and convolutional models close to the Transformer architecture, but not using self-attention. |
FlowMalTrans: Unsupervised Binary Code Translation for Malware Detection Using Flow-Adapter Architecture (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using deep learning to detect malware has attracted great attention due to its notable performance. |
| Approach: | a new approach uses Neural Machine Translation and Normalizing Flows to apply deep learning to malware detection. |
| Outcome: | The proposed approach reduces the burden of data collection by enabling malware detection across multiple ISAs. |
Contrastive Preference Learning for Neural Machine Translation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing discrepancies between token-level objective and overall sequence-level quality of a model are causing exposure bias and other issues in NMT. |
| Approach: | They propose a contrastive preference model that integrates an indicator function to fine-tune a pre-trained model in Neural Machine Translation. |
| Outcome: | The proposed model outperforms the traditional Plackett-Luce model on three language pairs and also outperFORMs token-level and sequence-level baseline models. |
Analyzing Challenges in Neural Machine Translation for Software Localization (2023.eacl-main)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is a new form of machine translation that reduces the post-editing time of human annotators. |
| Approach: | They propose to use a novel multilingual UI corpus collection to test NMT for user interfaces. |
| Outcome: | The proposed test set evaluates state-of-the-art methods on a UI translation task from English to German and identifies its limitations. |
Measuring and Mitigating Name Biases in Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Neural machine translation systems exhibit problematic biases, such as stereotypical gender bias in occupation terms. |
| Approach: | They propose a method to reduce biases in person name translations by randomly switching entities during translation. |
| Outcome: | The proposed method eliminates the problem without any effect on translation quality. |
Contrastive Token Learning with Similarity Decay for Repetition Suppression in Machine Translation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is pivotal for crosslingual conversation and trade . traditional solutions that penalize text redundancy or token reoccurrence have shown limited efficacy . |
| Approach: | They propose an algorithm that modulates suppression of tokens dynamically, informed by attention weights and inter-token distances. |
| Outcome: | The proposed algorithm outperforms existing methods in precision and generalizability. |
Non-Parametric Adaptation for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Neural Networks trained with gradient descent are susceptible to catastrophic forgetting due to parameter shift during the training process. |
| Approach: | They propose a semi-parametric approach that relies on local phrase level similarities to retrieve neighboring phrases that are useful for translation even when overall sentence similarity is low. |
| Outcome: | The proposed approach performs well on a heterogeneous dataset with WMT, IWSLT, JRC-Acquis and OpenSubtitles. |
CSP:Code-Switching Pre-training for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train neural machine translation models are data-hungry and low-resource . et al., 2018; Radford e.t., 2019; Yang ee.,2019) proposes a new pre-training method for NMT . |
| Approach: | They propose a new pre-training method which randomly replaces some words in the input sentence with their translation words in target language. |
| Outcome: | The proposed method improves on unsupervised and supervised NMT models by making full use of monolingual corpora. |
WIND: Weighting Instances Differentially for Model-Agnostic Domain Adaptation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for instance weighting cannot learn the weights which make the model generalize well in target domain. |
| Approach: | They propose a modelagnostic instance weighting algorithm which can learn the instance weights instead of manually designed weighting metrics. |
| Outcome: | The proposed method can learn the instance weights instead of manually designed weighting metrics. |
Understanding and Addressing the Under-Translation Problem from the Perspective of Decoding Objective (2024.acl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) has made remarkable progress over the past years, but under-translation and over-translatation remain challenging obstacles faced by NMT systems. |
| Approach: | They propose to employ the confidence of predicting the end of sentence (EOS) as a detector for under-translation and strengthen the confidence-based penalty to penalize candidates with a high risk of under-translated. |
| Outcome: | The proposed method can detect and rectify under-translated outputs, with minor impact on other correct translations. |
Context-aware Neural Machine Translation with Mini-batch Embedding (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing models that translate sentences in sentence-by-sentence manner do not consider the inter-sentent context in a language translation model. |
| Approach: | They propose to use mini-batch embedding to represent the features of sentences in a mini-Batch by choosing sentences from the same document and then embedd it into the model. |
| Outcome: | The proposed method outperforms baseline models and improves writing style or terminology to fit the document’s context. |
Chinese-Portuguese Machine Translation: A Study on Building Parallel Corpora from Comparable Texts (L18-1)
Copied to clipboard
| Challenge: | Chinese and Portuguese are very populous languages, but there is not much parallel corpora in the Chinese-Portuguese language pair. |
| Approach: | They propose to curate Chinese-Portuguese parallel corpora and evaluate their quality . they extract bilingual data from government websites and use Phrased-Based Machine Translation (PBMT) and Neural Machine Translation models to build large corpus. |
| Outcome: | The proposed method can be used as a benchmark for future Chinese-Portuguese MT systems. |
Isometric Neural Machine Translation using Phoneme Count Ratio Reward-based Reinforcement Learning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Traditional Automatic Video Dubbing (AVD) pipelines use isometric-NMT algorithms to regulate the length of the output text. |
| Approach: | They propose an isometric-NMT system that regulates the length of the output text . they propose a phoneme Count Compliance score to measure length compliance . |
| Outcome: | The proposed approach improves phoneme count compliance scores by 36% compared to state-of-the-art models in English-Hindi language pairs. |
Target Conditioning for One-to-Many Generation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Neural Machine Translation models lack diversity in their generated translations, even when paired with search algorithm, like beam search. |
| Approach: | They propose to model one-to-many mapping by conditioning a decoder on a latent variable that represents the domain of target sentences. |
| Outcome: | The proposed method can scale to any number of domains without affecting performance or training time. |
Neural Machine Translation with Heterogeneous Topic Knowledge Embeddings (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing NMT models can handle meaning ambiguities based on local contexts, but it remains a challenge to translate words in implicit collocations. |
| Approach: | They propose heterogeneous ways of embedding topic information into NMT models . they propose to incorporate topic knowledge embeddable into the NMT model . |
| Outcome: | The proposed methods outperform baselines on English -> German and English => French translation tasks. |
A Self-Distillation Recipe for Neural Machine Translation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for Neural Machine Translation (NMT) have been proven effective in improving the performance of computer vision tasks without pre-training a teacher. |
| Approach: | They propose a rank-order augmented Pearson correlation loss and an iterative distillation method to prevent the discrepancy of predictions between the student and a stronger teacher from disturbing the training. |
| Outcome: | The proposed method can lead to significant improvements over the strong Transformer baseline on low/middle/high-resource tasks, obtaining comparable or better performance with fewer layers. |
Handling Rare Word Problem using Synthetic Training Data for Sinhala and Tamil Neural Machine Translation (L18-1)
Copied to clipboard
| Challenge: | Lack of parallel training data influences rare word problem in Neural Machine Translation systems, especially for underresourced languages. |
| Approach: | They propose to use Parts of Speech tagging and morphological analysis as syntactic features to prune generated synthetic sentence pairs that do not adhere to language syntax. |
| Outcome: | The proposed methods show that they can prune sentences that do not adhere to language syntax over Sinhala to Tamil and Tamil to Sinhalak translation systems. |
AugVic: Exploiting BiText Vicinity for Low-Resource NMT (2021.findings-acl)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) systems often exhibit poor performance due to the lack of large bitext training corpora in low-resource languages. |
| Approach: | They propose a data augmentation framework which exploits the vicinal samples of the given bitext without using extra monolingual data explicitly. |
| Outcome: | The proposed framework can diversify in-domain bitext data with finer level control on four low-resource language pairs. |
Prevent the Language Model from being Overconfident in Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation models are based on partial translation and a language model that predicts the next token based only on partial. |
| Approach: | They propose a Margin-based Token-level Objective and a Sentence-level Goal to maximize the Margin . they propose to model the next token based on partial translation . |
| Outcome: | The proposed approach improves translation adequacy and fluency on English-to-German, Chinese-to English and French translation tasks. |
Adversarial Grammatical Error Correction (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that adversarial-GEC can achieve competitive GEC quality compared to NMT-based baselines. |
| Approach: | They propose an adversarial approach to Grammatical Error Correction using a transformer-based model and a sentence-pair classification model. |
| Outcome: | The proposed approach achieves competitive GEC quality compared to baselines. |
Sometimes We Want Ungrammatical Translations (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in Neural Machine Translation (NMT) systems focus on improving translation quality and improving robustness to perturbations. |
| Approach: | They propose a way to quantify faithfulness to the original text by focusing on word-order perturbations. |
| Outcome: | The proposed method aims to measure faithfulness and robustness in word-order perturbations without deleting or injecting tokens. |
Deconvolution-Based Global Decoding for Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Existing models for Neural Machine Translation (NMT) use Recurrent Neural Network (RNN) to generate translation word by word following a sequential order. |
| Approach: | They propose a Neural Machine Translation (NMT) model that decodes the sequence with the guidance of its structural prediction of the target-side context. |
| Outcome: | The proposed model is more competitive compared with the state-of-the-art methods and reduces repetition with the instruction from the target-side context for decoding. |
On Long-Tailed Phenomena in Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | State-of-the-art Neural Machine Translation models struggle with generating low-frequency tokens, tackling which remains a major challenge. |
| Approach: | They propose a loss function to better adapt model training to structural dependencies of conditional text generation by incorporating inductive biases of beam search into the training process. |
| Outcome: | The proposed method leads to significant gains over cross-entropy across different language pairs, especially on the generation of low-frequency words. |
Reference Network for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) generates translations in isolation, resulting in translation inconsistency and ambiguity. |
| Approach: | They propose to incorporate referring process into translation decoding of NMT by using local coordinates coding to obtain global context vectors containing monolingual and bilingual contextual information. |
| Outcome: | The proposed model improves translation quality with lightweight computation cost on Chinese-English and English-German translation tasks. |
Point, Disambiguate and Copy: Incorporating Bilingual Dictionaries for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to incorporate bilingual dictionaries into Neural Machine Translation (NMT) models have been criticized for lack of integration of bilingual lexical information into the neural architecture. |
| Approach: | They propose a neural architecture to incorporate bilingual dictionaries into Neural Machine Translation models by introducing three new components: Pointer, Disambiguator, and Copier. |
| Outcome: | The proposed method achieves the following merits inherently compared with previous efforts: (1) Pointer leverages the semantic information from bilingual dictionaries, for the first time, to better locate source words whose translation in dictionary can potentially be used; (2) Disambiguator synthesizes contextual information from source view and target view, both of which contribute to distinguishing translation of a specific source word from multiple candidates in dicaries; (3) Copier systematically connects Pointer and Disambiguators based on a hierarchical |
Token-wise Curriculum Learning for Neural Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing curriculum learning approaches to Neural Machine Translation (NMT) require sampling sufficient amounts of “easy” samples from training data at the early stage of training. |
| Approach: | They propose a token-wise curriculum learning approach that creates sufficient amounts of easy samples from training data. |
| Outcome: | The proposed approach outperforms baselines on five language pairs on low-resource languages. |
Neural Machine Translation of Text from Non-Native Speakers (N19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) systems are known to degrade when confronted with noisy data. |
| Approach: | They propose to augment training data with sentences containing artificially-introduced grammatical errors to make the system more robust to such errors. |
| Outcome: | The proposed approach recovers 1.0 BLEU out of 2.4 BLUE lost due to grammatical errors on a set of Spanish translations of the JFLEG grammar error correction corpus. |
Transforming Term Extraction: Transformer-Based Approaches to Multilingual Term Extraction Across Domains (2021.findings-acl)
Copied to clipboard
| Challenge: | Automated Term Extraction (ATE) is a challenging task, with few exceptions. |
| Approach: | They propose to use a transformer-based term extraction model to extract terms from sentences . they also propose to employ a language model for token classification and a sequence model to reduce sentences to terms . |
| Outcome: | The proposed models outperform baselines on the ATE challenge TermEval 2020 dataset in English, French, and Dutch. |
Generating Authentic Adversarial Examples beyond Meaning-preserving with Doubly Round-trip Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to generate adversarial examples for NMT use the meaning-preserving restriction. |
| Approach: | They propose a new definition for adversarial examples based on the Doubly Round-Trip Translation (DRTT) they introduce masked language models to construct bilingual adversarials based upon DRTT . |
| Outcome: | The proposed approach significantly improves the robustness of the NMT model on clean and noisy test sets. |
Dual-teacher Knowledge Distillation for Low-frequency Word Translation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation models are trained on parallel corpora with unbalanced word frequency distribution, resulting in high-frequency words being ignored. |
| Approach: | They propose to employ a low-frequency teacher model that excels in translating low- frequency words to guide the learning of the student model. |
| Outcome: | The proposed method achieves +0.64 BLEU improvements over the state-of-the-art method on the low-frequency translation task while maintaining the translation quality of high-frequency words. |
Incorporating Noisy Length Constraints into Transformer with Length-aware Positional Encodings (2020.coling-main)
Copied to clipboard
| Challenge: | Neural Machine Translation suffers from an under-translation problem due to limited modeling of output sequence lengths. |
| Approach: | They propose a method to train a Transformer model using length constraints based on positional encoding. |
| Outcome: | The proposed method outperforms a vanilla Transformer in an English-to-Japanese translation by 3.22 points . the noise injection improved robustness for length prediction errors, especially within the window size. |
Improving Target-side Lexical Transfer in Multilingual Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual data is more beneficial for NMT models that translate from the LRL to a target language than those that translate into the LLLs. |
| Approach: | They propose a decoder that embeds character n-grams into NMT models that translate from an LRL to a target language. |
| Outcome: | The proposed decoder improves the performance of NMT models that translate from an LRL to a target language. |
Document-Level Neural Machine Translation with Hierarchical Attention Networks (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) can be improved by including document-level contextual information. |
| Approach: | They propose a hierarchical attention model that captures document-level contextual information and conditioning on the NMT model’s own hidden states. |
| Outcome: | The proposed model improves the BLEU score over a strong NMT baseline with the state-of-the-art in context-aware methods and that both the encoder and decoder benefit from context in complementary ways. |
Synchronously Generating Two Languages with Interactive Decoding (D19-1)
Copied to clipboard
| Challenge: | Experimental results show that multilingual NMT models handle multiple language pairs in one model. |
| Approach: | They propose an interactive approach to translate a source language into two different languages simultaneously and interactively. |
| Outcome: | The proposed approach improves on IWSLT and WMT datasets. |
Learning When to Concentrate or Divert Attention: Self-Adaptive Attention Temperature for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation models treat decoding at each time step equally with the same matrix . conventional methods treat decoder outputs at all time steps with the identical weight matrix causing inaccuracy . |
| Approach: | They propose a model with a mechanism to control the softness of attention by means of an attention temperature. |
| Outcome: | The proposed model outperforms baseline models on Chinese-English and English-Vietnamese translations. |
Building a Task-oriented Dialog System for Languages with no Training Data: the Case for Basque (2020.lrec-1)
Copied to clipboard
| Challenge: | a new approach to task-oriented dialog systems is proposed for less-resourced languages . the authors compare training data projection with direct model transfer strategies . |
| Approach: | They propose to project existing annotations in rich-resource languages and use NMT to project them. |
| Outcome: | The proposed system outperforms models trained on Basque projected data in intent classification and slot filling tasks. |
Integrating Domain Terminology into Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work on terminology integration into Neural Machine Translation shows it can dynamically specialize translation to a specific domain. |
| Approach: | They extend existing work on terminology integration into Neural Machine Translation . they use placeholders complemented by morphosyntactic annotation to integrate terminology . |
| Outcome: | The proposed method surpasses the surface generalization shown by other techniques. |
Continuous Language Generative Flow (2021.acl-long)
Copied to clipboard
| Challenge: | Recent years have witnessed various types of generative models for natural language generation (NLG), especially RNNs or transformers. |
| Approach: | They propose a flow-based language generation model that adapts flow-derived generative models to language generation via continuous input embeddings, adapted affine coupling structures, and a novel architecture for autoregressive text generation. |
| Outcome: | The proposed model improves on QG and NMT and improves performance over baselines on SQuAD and TVQA and NML16. |
Reducing Disambiguation Biases in NMT by Leveraging Explicit Word Sense Information (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that Neural Machine Translation models struggle to disambiguate polysemous words without lapsing into their most frequent senses. |
| Approach: | They propose a way to automatically create high-precision sense-annotated parallel corpora . they then propose 'fine-tuning' strategies to exploit these sense annotations during training . |
| Outcome: | The proposed approach achieves higher BLEU scores than its vanilla counterpart in 3 language pairs. |
WikiLingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarization (2020.findings-emnlp)
Copied to clipboard
| Challenge: | a lack of high quality multilingual data for cross-lingual summarization is a costly endeavor since it requires humans to read, comprehend, condense, and paraphrase entire articles. |
| Approach: | They propose to use a large-scale, multilingual dataset to evaluate cross-lingual abstractive summarization systems. |
| Outcome: | The proposed method significantly outperforms baseline approaches while being more cost efficient during inference. |
Improving Machine Translation with Phrase Pair Injection and Corpus Filtering (2022.emnlp-main)
Copied to clipboard
| Challenge: | In this paper, we show that the combination of Phrase Pair Injection and Corpus Filtering boosts the performance of Neural Machine Translation systems. |
| Approach: | They propose to combine Phrase Pair Injection and Corpus Filtering to boost performance of Neural Machine Translation systems. |
| Outcome: | The proposed method improves machine translation models on low-resource language pairs . BLEU score improves over models trained with whole pseudo-parallel corpus augmented with parallel corpus. |
Towards Inducing Long-Context Abilities in Multilingual Neural Machine Translation Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation models traditionally use Sinusoidal Positional Embeddings . retraining with newer methods like ROPE or ALIBI is computationally expensive . |
| Approach: | They propose to transition NMT models from Sinusoidal to Relative PEs without compromising performance. |
| Outcome: | The proposed approach outperforms models trained with Sinusoidal PEs on document-level benchmarks . the results show that parameter-efficient fine-tuning can facilitate the transition . |
Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Memorization presents a challenge for constrained Natural Language Generation tasks . previous studies focused on counterfactual memorization, linking it to hallucinations . |
| Approach: | They propose an algorithm for extractive memorization in constrained sequence generation tasks . they propose to elicit non-memorized translations of memorized samples from the same model . |
| Outcome: | The proposed algorithm could be leveraged to mitigate memorization in the model through finetuning. |
SALTED: A Framework for SAlient Long-tail Translation Error Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Traditional machine translation metrics are insensitive to the long tail of behavioral problems. |
| Approach: | They propose a specification-based framework for behavioral testing of NMT models . they use high-precision detectors that flag errors between a source sentence and output . |
| Outcome: | The proposed framework provides a reliable view of problems that were previously invisible. |
Addressing word-order Divergence in Multilingual Neural Machine Translation for extremely Low Resource Languages (N19-1)
Copied to clipboard
| Challenge: | Existing studies show that transfer learning works best when the languages are related. |
| Approach: | They propose to pre-order assisting language sentences to match the word order of the source language and train the parent model. |
| Outcome: | The proposed model can improve translation quality in low-resource scenarios by pre-ordering the assisting language sentences to match the word order of the source language and training the parent model. |
DC-MBR: Distributional Cooling for Minimum Bayesian Risk Decoding (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for decoding target language are degenerate, hallucinating or empty. |
| Approach: | They propose a method that tunes down the Softmax temperature to reduce autoregressive over-smoothness by label smoothing the output distributions. |
| Outcome: | The proposed method improves MBR in various settings. |
F-MALLOC: Feed-forward Memory Allocation for Continual Learning in Neural Machine Translation (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to address Catastrophic Forgetting (CF) have been developed to avoid forgetting and maintain system extensibility. |
| Approach: | They propose a method to reduce Catastrophic Forgetting (CF) by decomposing feed-forward layers into discrete memory cells and ensuring robust extendability. |
| Outcome: | The proposed method achieves higher BLEU scores and almost zero forgetting while maintaining robust extendability. |
Unsupervised Statistical Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) systems can be trained from monolingual corpora without supervision. |
| Approach: | They propose a phrase-based approach that trains from monolingual corpora . their method is based on phrase-driven Statistical Machine Translation (SMT) they propose to train NMT systems without supervision from monolinguistic corpors . |
| Outcome: | The proposed approach improves on the existing supervised systems by combining a phrase table with an n-gram language model and fine-tuning hyperparameters through an unsupervised MERT variant. |
Bridging the Gap between Training and Inference for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation generates target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Approach: | They propose to use ground truth and inference to generate target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Outcome: | Experiments on Chinese->English and WMT’14 English->German translation tasks show that the proposed model can achieve significant improvements on multiple datasets. |
Universal Conditional Masked Language Pre-training for Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Pre-trained sequence-to-sequence models have significantly improved Neural Machine Translation (NMT) this paper demonstrates that pre-training a sequence- to-squence model with a bidirectional decoder can produce notable performance gains for both Autoregressive and Non-autoregressive NMT tasks. |
| Approach: | They propose a conditional masked language model pre-trained on bilingual and monolingual corpora in many languages. |
| Outcome: | The proposed model can achieve significant performance improvements on all scenarios from low- to extremely high-resource languages. |
Multi-level Community-awareness Graph Neural Networks for Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Recent studies have used Graph Neural Networks (GNNs) to encode language knowledge into token embeddings. |
| Approach: | They propose a multi-level community-awareness Graph Neural Network layer to jointly model local and global relationships between words and their linguistic roles in multiple communities. |
| Outcome: | The proposed method reduces time complexity in very long sentences while preserving the original meaning. |
Speeding up Transformer Decoding via an Attention Refinement Network (2022.coling-1)
Copied to clipboard
| Challenge: | Extensive experiments on ten WMT machine translation tasks show that the proposed model yields an average of 1.35x faster (with almost no decrease in BLEU) |
| Approach: | They propose a weighted residual network which reconstructs attention by reusing the features across layers. |
| Outcome: | The proposed model is 1.35x faster than the state-of-the-art inference model on translation tasks compared to AAN and SAN models with fewer parameter numbers . |
Linguistically Informed Hindi-English Neural Machine Translation (2020.lrec-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) is a promising approach to machine translation . lack of parallel training data for Hindi-English is limiting . |
| Approach: | They propose to incorporate linguistic knowledge encoded by Hindi phenomena into a Transformer model to improve the translation performance. |
| Outcome: | The proposed model incorporates linguistic features to improve the translation performance. |
MuST-Cinema: a Speech-to-Subtitles corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for subtitling are laborious and costly, says aaron sanchez . he says the current methods are laboriously complex and require manual work . |
| Approach: | They propose to use TED subtitles to build a multilingual speech translation corpus . they propose to annotate existing subtitling corpora with subtitle breaks . |
| Outcome: | The proposed model can be used to segment sentences into subtitles and reduces human work . the proposed model reduces the time and cost of human subtitling tasks . |
To Case or not to case: Evaluating Casing Methods for Neural Machine Translation (2020.lrec-1)
Copied to clipboard
| Challenge: | Comparative evaluation of casing methods for Neural Machine Translation . evaluators evaluated methods for tokenisation and word segmentation into subword units . |
| Approach: | They evaluate three main casing methods for Neural Machine Translation to determine optimal handling of capitalisation. |
| Outcome: | The proposed methods are used to handle capitalisation on English-German and English-Turkish datasets. |
Handle with Care: A Case Study in Comparable Corpora Exploitation for Neural Machine Translation (2020.lrec-1)
Copied to clipboard
| Challenge: | Comparable corpora are an important source of potential parallel data, suitable for training data-driven machine translation systems. |
| Approach: | They present a case study on the exploitation of comparable corpora for machine translation. |
| Outcome: | The results show that filtering in terms of alignment thresholds and length-difference outliers has a significant impact on translation quality. |
Multiword Expression aware Neural Machine Translation (2020.lrec-1)
Copied to clipboard
| Challenge: | Multiword Expressions (MWEs) are a pervasive phenomenon in all natural languages and challenge NLP applications because of their unpredictable morpho-syntactic and lexico--semantic behaviour. |
| Approach: | They propose to use linguistic resources to improve MWE translation and MWE generation by up to 5.09 BLEU points on MWE test sets. |
| Outcome: | The proposed annotation and data augmentation improve translation quality and increase performance by up to 5.09 BLEU points on MWE test sets. |
Hypoformer: Hybrid Decomposition Transformer for Edge-friendly Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to compress Transformer are limited to sub-components, e.g., selfattention networks or embedding layer. |
| Approach: | They propose a Hybrid Tensor-Train decomposition which retains full rank and meanwhile reduces operations and parameters. |
| Outcome: | The proposed model outperforms light-weight SOTA methods on three translation tasks and achieves 7.1 points absolute improvement in BLEU and 1.27 X speedup on IWSLT’14 De-En task. |
Selective Knowledge Distillation for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation models achieve state-of-the-art performance on many translation benchmarks. |
| Approach: | They propose a protocol that analyzes different impacts of samples by comparing various samples’ partitions. |
| Outcome: | The proposed methods yield up to +1.28 and +0.89 BLEU points improvements over the Transformer baseline, respectively. |
GATE X-E : A Challenge Set for Gender-Fair Translations from Weakly-Gendered Languages (2024.findings-acl)
Copied to clipboard
| Challenge: | despite improvements in NMT quality and adoption, perpetuation of gender bias remains a concern. |
| Approach: | They propose a translation gender rewriting solution built with GPT-4 and use it to evaluate it. |
| Outcome: | The proposed translation gender rewriting solution is open source and can be used in English. |
A Tree-based Decoder for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Existing work on adding syntactic information to NMT systems is limited to linguistically-inspired tree structures. |
| Approach: | They propose an NMT model that can naturally generate the topology of an arbitrary tree structure on the target side. |
| Outcome: | The proposed model outperforms standard seq2seq models by 2.1 BLEU points and other methods for incorporating target-side syntax by 0.7 BLUE points. |
Exploring Recombination for Efficient Decoding of Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) decoder captures features of entire prediction history . some partial hypotheses with different prefixes will be regarded differently no matter how similar they are . |
| Approach: | They propose a method that uses a n-gram suffix to adapt it to beam search decoding. |
| Outcome: | The proposed method can obtain similar translation quality with a smaller beam size, making it more efficient. |
PheMT: A Phenomenon-wise Dataset for Machine Translation Robustness on User-Generated Contents (2020.coling-main)
Copied to clipboard
| Challenge: | Existing studies suggest that Neural Machine Translation still struggles with certain kinds of input with considerable noise, such as User-Generated Contents (UGC) on the Internet. |
| Approach: | They propose to evaluate the robustness of Neural Machine Translation models against specific linguistic phenomena in Japanese-English translation. |
| Outcome: | The proposed model can handle user-generated content (UGC) on the Internet, but it is difficult to translate clean inputs. |
Evaluating Pre-training Objectives for Low-Resource Translation into Morphologically Rich Languages (2022.lrec-1)
Copied to clipboard
| Challenge: | a lack of parallel data is a major limitation for Neural Machine Translation systems, especially for morphologically rich languages. |
| Approach: | They propose to leverage target monolingual data to overcome the lack of parallel data . they introduce a new technique called PT-Inflect to train NMT systems . |
| Outcome: | The proposed techniques outperform NMT systems trained on parallel data on four typologically diverse target languages. |
AdvAug: Robust Adversarial Augmentation for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work in neural machine translation has led to dramatic improvements in both research and commercial systems. |
| Approach: | They propose a adversarial augmentation method for Neural Machine Translation that minimizes vicinal risk over virtual sentences . they use a novel vicinity distribution for adversarials to describe a smooth interpolated embedding space . |
| Outcome: | The proposed method outperforms the current method on Chinese-English, English-French, and English-German translation benchmarks. |
DivEMT: Neural Machine Translation Post-Editing Effort Across Typologically Diverse Languages (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in neural language modeling and multilingual training have prompted widespread adoption of machine translation (MT) technologies across an unprecedented range of world languages. |
| Approach: | They propose to use a dataset to assess the impact of two state-of-the-art NMT systems, Google Translate and the multilingual mBART-50 model, on translation productivity. |
| Outcome: | The proposed model is faster than translation from scratch, but the magnitude of productivity gains varies widely across systems and languages. |
Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Prior work treats all types of mismatches between source and target as noise . Consequently, it remains unclear how noisy parallel training samples impact NMT training. |
| Approach: | They propose a divergent-aware NMT framework that uses factors to help NMT recover from the degradation caused by naturally occurring divergences. |
| Outcome: | The proposed framework improves translation quality and model calibration on EN-FR tasks. |
Neural Machine Translation with Monolingual Translation Memory (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work has shown that Translation Memory (TM) can boost the performance of Neural Machine Translation (NMT) |
| Approach: | They propose a framework that uses monolingual memory and performs learnable memory retrieval in a cross-lingual manner. |
| Outcome: | The proposed framework outperforms strong TM-augmented NMT baselines using bilingual TM and outperformed existing models in low-resource and domain adaptation scenarios. |
AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation (2023.findings-acl)
Copied to clipboard
Ganesh Jawahar, Subhabrata Mukherjee, Xiaodong Liu, Young Jin Kim, Muhammad Abdul-Mageed, Laks Lakshmanan, V.S., Ahmed Hassan Awadallah, Sebastien Bubeck, Jianfeng Gao
| Challenge: | Existing MoE designs do not consider computational constraints (e.g., FLOPs, latency) Existing works in MoE consider homogeneous design where the same number of experts of the same size are placed uniformly throughout the network. |
| Approach: | They propose a framework for designing heterogeneous MoEs under computational constraints. |
| Outcome: | The proposed framework achieves 4x inference speedup and FLOPs reduction over manual models and within 1 BLEU point of MoE SwitchTransformer over benchmark datasets for NMT. |
Integrating Multi-scale Contextualized Information for Byte-based Neural Machine Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Subword tokenization is a common method for vocabulary building in NMT systems . but, it has its disadvantages, such as a vocabulary cannot be modified once it is learned . |
| Approach: | They propose a method that learns contextualized information of varying scales . they propose byte-based tokenization to solve these problems with few embedding parameters . |
| Outcome: | Experiments show that the proposed method outperforms subword-based methods in multilingual and out-of-domain scenarios. |
Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformer (2022.emnlp-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) relies on source sentence and target prefix attributions for each input token. |
| Approach: | They propose an interpretability method that tracks input tokens’ attributions for both contexts and extends it to any encoder-decoder Transformer-based model. |
| Outcome: | The proposed method can be extended to any encoder-decoder Transformer-based model and provides insights into their behaviour. |
Progressive Translation: Improving Domain Robustness of Neural Machine Translation with Intermediate Sequences (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that intermediate supervision signals benefit various tasks such as math word problems and commonsense reasoning. |
| Approach: | They propose to introduce an inductive bias that reflects a domain-agnostic principle of translation and a full-permutation multi-task learning to alleviate spurious correlations. |
| Outcome: | The proposed signals reduce spurious correlations and spurious hallucinations on out-of-domain translation, and are especially promising in low-resource scenarios. |
Document Translation vs. Query Translation for Cross-Lingual Information Retrieval in the Medical Domain (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies of document translation and query translation are outdated and do not reflect the current advances in machine translation. |
| Approach: | They compare document translation and query translation approaches to cross-lingual information retrieval . they exploit Statistical Machine Translation and Neural Machine Translation paradigms to translate queries into English and English . |
| Outcome: | The proposed approach outperforms the DT approach in translation quality and retrieval quality. |
WebNLG-IT: Construction of an aligned RDF-Italian corpus through Machine Translation techniques (2025.findings-acl)
Copied to clipboard
| Challenge: | Using NMT and hand-written rules, we created the first aligned Italian RDF-to-text corpus . |
| Approach: | They propose to use NMT to create an Italian version of the WebNLG corpus and to refine and improve the quality of the produced resource. |
| Outcome: | The proposed system is the best on the original English version and the best in the second step, it improves and refines the quality of the produced resource. |
A Multi-Task Architecture on Relevance-based Neural Query Translation (P19-1)
Copied to clipboard
| Challenge: | Existing models for cross-lingual information retrieval are not aware of the vocabulary distribution of the retrieval corpus. |
| Approach: | They propose a multi-task learning approach to train a Neural Machine Translation model with a Relevance-based Auxiliary Task (RAT) for search query translation. |
| Outcome: | The proposed model achieves 16% improvement over a strong baseline on Italian-English query-document dataset. |
Vector-Vector-Matrix Architecture: A Novel Hardware-Aware Framework for Low-Latency Inference in NLP Applications (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve accuracy of neural networks are slow due to computational complexity. |
| Approach: | They propose a vector-vector-matrix architecture which greatly reduces latency at inference time for NLP applications by a factor of four. |
| Outcome: | The proposed framework reduces the latency of sequence-to-sequence and Transformer models used for NMT by a factor of four. |
Effectively pretraining a speech translation decoder with Machine Translation data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve the performance of AST systems are based on pretraining the encoder parameters using an ASR model, but using a pretrained MT decoder is not beneficial or improves the results. |
| Approach: | They propose to use an adversarial regularizer to bring the encoder representations of the ASR and NMT tasks closer even though they are in different modalities. |
| Outcome: | The proposed model can be pre-trained using the Automatic Speech Recognition (ASR) task even in different languages and improves in low resource settings. |
Multi-Sentence Resampling: A Simple Approach to Alleviate Dataset Length Bias and Beam-Search Degradation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Neural Machine Translation suffers from a beam-search problem after a certain point, especially for long sentences. |
| Approach: | They propose a data augmentation technique that concatenates several sentences from the original dataset to make a long training example. |
| Outcome: | The proposed technique significantly reduces degradation with growing beam size and improves translation quality on the IWSTL15 En-Vi, IWStl17 En-Fr, and WMT14 En-De datasets. |
Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem (2020.acl-main)
Copied to clipboard
| Challenge: | Training data for NLP tasks often exhibits gender bias in that fewer sentences refer to women than to men. |
| Approach: | They propose a lattice-rescoring scheme which allows a trade-off between general translation quality and bias reduction during adaptation and inference time. |
| Outcome: | The proposed approach outperforms all systems evaluated on WinoMT with no degradation of general test set BLEU. |
Using Context in Neural Machine Translation Training Objectives (2020.acl-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) training is based on document-level metrics, not sentence-level BLEU. |
| Approach: | They propose to merge document-level metrics with batch-level documents to improve NMT training. |
| Outcome: | The proposed training is more robust for document-level metrics than sequence MRT and maximum-likelihood training. |
Domain adapted machine translation: What does catastrophic forgetting forget and why? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) models can be specialized by domain adaptation, often fine-tuning on a dataset of interest. |
| Approach: | They propose a novel approach to understanding catastrophic forgetting during NMT adaptation by investigating the relationship between the data and the in-domain vocabulary coverage. |
| Outcome: | The proposed model can be specialized by fine-tuning on a domain of interest, but can fail to achieve the predicted quality of the target domain. |
BERTology for Machine Translation: What BERT Knows about Linguistic Difficulties for Translation (2022.lrec-1)
Copied to clipboard
| Challenge: | Pre-trained transformer-based models have shown excellent performance in most benchmark tests, but lack a good understanding of the linguistic knowledge of BERT in Neural Machine Translation (NMT). |
| Approach: | They propose to use QE models to analyze BERT's syntactic dependencies and their impact on machine translation quality. |
| Outcome: | The proposed model is able to model with self-attention in the pre-training phase, which improves generalization ability. |
Exploiting Target Language Data for Neural Machine Translation Beyond Back Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) encounters challenges when translating in new domains and low-resource languages. |
| Approach: | They propose a variant of k-nearest neighbor machine translation that utilizes target language data by constructing a pseudo datastore. |
| Outcome: | The proposed method exhibits strong domain adaptation capability in both high-resource and low-resourced machine translation. |
Evaluating Robustness to Input Perturbations for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work has shown that Neural Machine Translation models are brittle to small perturbations in the input. |
| Approach: | They propose to use subword regularization to measure the relative degradation and changes in translation when perturbations are added to the input. |
| Outcome: | The proposed measures show that the models are more robust to perturbations when subword regularization methods are used. |
Leveraging GPT-4 for Automatic Translation Post-Editing (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Neural Machine Translation models still require translation post-editing to rectify errors and enhance quality under critical settings. |
| Approach: | They use GPT-4 to automatically post-edit NMT outputs across several language pairs . they show that GPT4 is adept at translation post- editing, producing meaningful edits . |
| Outcome: | The proposed translation post-editor improves on state-of-the-art language models on English-Chinese, English-German, Chinese-English and German-English language pairs. |
Efficient Domain Adaptation for Non-Autoregressive Machine Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing non-parametric approaches like nearest neighbor machine translation have made small Autoregressive translation models less efficient . despite their impressive generalization and task performance, LLMs suffer from prohibitive inference cost when confronted with specific domains. |
| Approach: | They propose a domain adaptation approach that tailors a k-nearest-neighbor algorithm for NAT models that incorporates the parallel nature of NAT. |
| Outcome: | The proposed approach achieves significant improvements over the Base-NAT model and exhibits enhanced efficiency. |
Code-Switching with Word Senses for Pretraining in Neural Machine Translation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Lexical ambiguity is a significant and pervasive challenge in Neural Machine Translation (NMT) many state-of-the-art (SOTA) NMT systems struggle to handle polysemous words . |
| Approach: | They propose an end-to-end approach for pretraining multilingual NMT models leveraging word sense-specific information from Knowledge Bases. |
| Outcome: | The proposed approach improves translation quality and scales to various data and resource-strapped scenarios. |
Does Context Help Mitigate Gender Bias in Neural Machine Translation? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation models perpetuate gender bias in their training data distribution. |
| Approach: | They examine the gender bias in Neural Machine Translation by using context-aware models to enhance translation accuracy for feminine terms and translation with non-informative context in Basque to Spanish. |
| Outcome: | The proposed models can maintain or even amplify gender bias in translations of stereotypical professions in English and with non-informative context in Basque to Spanish. |
Learning from Wrong Predictions in Low-Resource Neural Machine Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to Neural Machine Translation use additional linguistic sources and software tools but these are often not available in less favoured languages. |
| Approach: | They propose a pre-training strategy that leverages the relationships and similarities that exist between unaligned sentences to increase the dataset size of endangered and low-resource languages. |
| Outcome: | The proposed approach increases the dataset size of endangered and low-resource languages by the square of the initial quantity, matching the typical size of high-resourced datasets such as WMT14 En-Fr. |
Unsupervised Binary Code Translation with Application to Code Clone Detection and Vulnerability Discovery (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that binary code analysis is a key problem in software security research. |
| Approach: | They propose to apply Neural Machine Translation to binary code analysis . they translate a binary in a low-resource ISA and train a model on the high-resourced ISA . |
| Outcome: | The proposed model can be used to analyze binary code across ISAs using a high-resource ISA. |
Improving the Quality of Web-mined Parallel Corpora of Low-Resource Languages using Debiasing Heuristics (2025.emnlp-main)
Copied to clipboard
| Challenge: | Parallel Data Curation (PDC) techniques aim to filter out noisy parallel sentences from web-mined corpora. |
| Approach: | They propose to rank parallel sentences using similarity scores on sentence embeddings derived from Pre-trained Multilingual Language Models (multiPLMs) . previous research has shown that the choice of multiPLM significantly impacts the quality of the filtered parallel corpus. |
| Outcome: | The proposed methods reduce disparities between multiPLMs while producing better results. |